Locating data in a set with a single index using multiple property values

ABSTRACT

Indexing objects in a datastore, where the objects are characterized by a set of object properties. For objects from a datastore: a Bloom filter for the object properties, based on a predetermined Bloom filter configuration, is created; the Hamming value of the Bloom filter is determined; a binary logarithm of the Bloom filter is calculated; and the Bloom filter, the Hamming value, the binary logarithm, and a representation of the object is stored in the datastore.

BACKGROUND

The present invention relates generally to retrieval of records storedin a database, and in particular to an index that facilitates efficientaccess to specific records.

A relational database is a digital database whose organization is basedon the relational model of data. This model organizes data into one ormore tables, or relations, of rows and columns, with a unique key foreach row. Generally, each entity type described in a database has itsown table, the rows representing instances of that type of entity, orobjects, and the columns representing values, or properties, attributedto that instance. Because each row in a table has its own unique key,rows in a table can be linked to rows in other tables by storing theunique key of the row to which it should be linked. Data relationshipsof arbitrary complexity can be represented using this set of concepts.The various software systems used to maintain relational databases areknown as Relational Database Management Systems (RDBMS). Virtually allrelational database systems use SQL (Structured Query Language) as thelanguage for querying and maintaining the database.

A database index is a data structure that improves the speed of dataretrieval operations on a database table at the cost of additionalwrites and storage space to maintain the index data structure. Indexesare used to quickly locate data in a database table without having toperform a full table scan, which searches every row in the table.Indexes associated with one or more columns of a database table canprovide a basis for both rapid random lookups and efficient access ofordered rows.

A Bloom filter is a memory-efficient, probabilistic data structure thatsupports approximate membership queries in a set. When testing whetheran object is a member of a set represented by a Bloom filter, a querymay return “definitely not in set” or “may be in set,” with a smallprobability of false positives. A Bloom filter is typically implementedas a bit vector, or array, into which a set of values, representing setelements, is hashed. In general, a Bloom filter may be considered whenspace is at a premium and the effect of false positives can bemitigated. Due to their efficiency, compact representation, andflexibility in allowing a trade-off between memory requirements andfalse positive probability, Bloom filters are popular in representingdiverse sets of data. For example, they are used in databases,distributed systems, web caching, and other network applications, wheresystems need to share information about what resources they have. Atypical example is using a Bloom filter to reduce expensive disk ornetwork lookups for non-existent objects. If the Bloom filter indicatesthat the object is not present, then an expensive lookup may be avoided;otherwise, a lookup may be performed, but it may fail a certainpercentage of the time.

A Bloom filter index organizes a collection of Bloom filters. Searchinga Bloom filter index for a target Bloom filter typically involvescomparing indexed Bloom filters with the target Bloom filter to findmatches. A standard, but inefficient, technique for locating a targetBloom filter in a collection of Bloom filters is to linearly search alist of all the Bloom filters in the collection for ones that match thetarget.

SUMMARY

Embodiments of the present invention disclose a computer-implementedmethod, computer program product, and computer system for indexingobjects in a datastore, wherein the objects are characterized by a setof object properties. For one or more objects from a datastore: a Bloomfilter for the object properties, based on a predetermined Bloom filterconfiguration is created; a Hamming value of the Bloom filter isdetermined; a binary logarithm of the Bloom filter is calculated; andthe Bloom filter, the Hamming value, the binary logarithm, and arepresentation of the object is stored in the datastore.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a system diagram of a database indexing system, in accordancewith an embodiment of the present invention.

FIG. 2 is a flowchart depicting operational steps of a database indexingprogram, in accordance with an embodiment of the present invention.

FIG. 3 is another flowchart depicting operational steps of a databaseindexing program, in accordance with an embodiment of the presentinvention.

FIG. 4 is another flowchart depicting operational steps of a databaseindexing program, in accordance with an embodiment of the presentinvention.

FIG. 5 is another flowchart depicting operational steps of a databaseindexing program, in accordance with an embodiment of the presentinvention.

FIG. 6 is another flowchart depicting operational steps of a databaseindexing program, in accordance with an embodiment of the presentinvention.

FIG. 7 is a functional block diagram illustrating a data processingenvironment, in accordance with an embodiment of the present invention.

DETAILED DESCRIPTION

In relational databases, multiple indexes may be created to facilitatesearching for various combinations of column values, or properties, ofthe data items, or objects, stored as rows in the database. Each indexincurs additional overhead when inserting rows in a database table. Rowsin a database are also referred to as records or tuples. In very largedata systems, the overhead involved in searching indexes defined for atable may be high enough that a full table scan is preferable. Moreover,for extremely large data systems, index maintenance may becomeintrusive. In certain cases, removing indexes may actually improvedatabase performance. A solution whereby a single index can readilydetect the presence of objects in a database, based on multipleproperties would, therefore, be advantageous.

A Bloom filter for a set of n elements uses an array, or bit vector, ofm bits, and a collection of k independent hash functions, for a conciserepresentation of the set. Several families of hash functions aresuitable for this purpose, for example, the freely availableMurmurHash3. The choice of m and k will depend on n and an acceptablefalse positive probability p, as described below. The size of a Bloomfilter is the length m of its bit vector. The k hash functionsindependently map each of the elements of the set to positions in thebit array, for example to positions with indices 1, . . . , m.Initially, a Bloom filter has all m bits in its array set to 0. To addan element of the set to the Bloom filter, each of the k hash functionsmaps the element to an array position and the bit in that position isset to 1. After each of the k hash functions has mapped each of the nelements to one of the m bit positions in the Bloom filter arrayrepresenting the set, to check whether the set contains a specificobject, the k hash functions are applied to the object. If any of theresulting k positions in the Bloom filter array contain a 0, the objectis definitely not present in the set. If all k positions contain a 1,the set might contain the object, i.e., the 1's might represent a truepositive or a false positive. A false positive may occur if the bitsrelated to a given object have been set to 1 by other objects. There isa trade-off between the size of a Bloom filter and the probability offalse positives returned by it. The probability of false positives canbe lowered by increasing the size of the Bloom filters used.

In what follows, the term Bloom filter will be used to refer both to thedata structure that includes both a bit vector, or array, and anassociated collection of hash functions, and the array itself, as iscommon practice. Whenever a collection of Bloom filters is referenced,it is understood that they are all of the same size and use the samecollection of hash functions. A particular choice of m, n, and k hashfunctions is referred to as a Bloom filter configuration.

In practice, once a Bloom filter representing a set has been created, tocheck whether a specific object is in the set, a target Bloom filterrepresenting the object is created by applying the k hash functions tothe object and setting the appropriate bits in the target Bloom filterto 1. The target Bloom filter is then compared to the Bloom filter forthe set. A match is found if each 1 bit in the target Bloom filtercorresponds to a 1 bit at the same position in the set's Bloom filter.In practice, this means that the result of performing a logical ANDoperation between the Bloom filter for the set and the target Bloomfilter equals the target Bloom filter. A match indicates that the objectis in the set, with probability depending on p.

As mentioned above, a Bloom filter is characterized by the number ofelements n in the set it represents, the number k of hash functionsused, the length m of its bit vector, and the probability p of falsepositives. It is known that the relationship between these properties isclosely approximated by equation (1):

$\begin{matrix}{p = ( {1 - e^{- \frac{kn}{m}}} )^{k}} & (1)\end{matrix}$

From this equation one may create a Bloom filter with desiredproperties, for example, by adjusting m and k to give an acceptableprobability p of false positives. For example, equation (1) implies thata Bloom filter representing a set of n=3 objects, with a false positiveprobability of around 1 in 100,000, may be created using a bit vectorwith m=72 bits and k=17 hash functions. A Bloom filter corresponding tosuch a set may be created by applying each of the 17 hash functions tothe three objects and setting the appropriate bits in the Bloom filterto 1. In practice, a collection of Bloom filters representing such a setwill all have the same size and share the same set of hash functions.

For example, one use of Bloom filters is to represent data in selectedcolumns of a database table. Based on the selected columns, a Bloomfilter configuration may be defined with appropriate values of m, k, n,and p, using equation (1). The integer n represents the number of columnvalues, or properties, selected. Equation (1) implies, for example, thata Bloom filter representing a row in the table with given values for thethree properties, with a false positive probability of around 1 in100,000, may be created using an array with m=72 bits and k=17 hashfunctions. A Bloom filter corresponding to such a row is created byapplying each of the 17 hash functions to the three properties andsetting the appropriate bits in the Bloom filter to 1.

Bloom filters possess various properties that may be utilized tofacilitate indexing. One property is that a Bloom filter created from asubset of a set matches a Bloom filter created from the full set.Another is that when matching a target Bloom filter, only Bloom filterswith at least as many 1 bits can match. Also, when the Bloom filter isconsidered as an unsigned integer, only Bloom filters having an arraywith an equal or higher numerical value can match. For this purpose, thebinary logarithm of the numerical value is a convenient substitute forthe actual numerical value, which may be extremely large. The binarylogarithm may be efficiently computed, for example, using a knownrecursive approximation technique on the Bloom filter. In what follows,binary log refers to a computed approximation to a binary logarithm.

Embodiments of the present invention are directed to Bloom filters thatindex data such as rows in a database. A Bloom filter is created foreach row, with appropriate bits set in the Bloom filter, each bitposition representing a column to be indexed. Associated with the Bloomfilter is a Hamming value, the number of bits in the Bloom filter thatare set to 1, and an approximation to the binary logarithm of the Bloomfilter considered as an unsigned integer. The binary logarithm and theHamming value provide a lower bound for a database search using theBloom filters. That is, when searching the database for an object withspecified properties, a target, or search, Bloom filter is created thatrepresents the columns to inspect, and the index is searched for matchesonly among Bloom filters whose Hamming values and binary logarithms aregreater than or equal to those of the target Bloom filter. This allows asingle index to be used to index multiple data columns, and allows theindex to be queried for any combination of exactly matched columnvalues, thus limiting the number of index scans required to find amatch.

FIG. 1 is a functional block diagram of a database indexing system 100,in accordance with an embodiment of the present invention. Databaseindexing system 100 includes computing device 110. Computing device 110represents the computing environment or platform that hosts databaseindexing program 112. In various embodiments, computing device 110 maybe a laptop computer, netbook computer, personal computer (PC), adesktop computer, or any programmable electronic device capable ofhosting database indexing program 112, in accordance with embodiments ofthe invention. Computing device 110 may include internal and externalhardware components, as depicted and described in further detail belowwith reference to FIG. 7.

In an exemplary embodiment of the invention, computing device 110includes database indexing program 112 and datastore 124.

Datastore 124 represents a store of data, in accordance with anembodiment of the present invention. For example, datastore 124 mayinclude one or more databases. Datastore 124 may also store an index ofdatabase objects, as described below. Datastore 124 may reside, forexample, on computer readable storage media 908 (FIG. 7).

Database indexing program 112 operates generally to create and maintainan index of Bloom filters representing objects stored as rows in adatabase table, in accordance with an embodiment of the invention. Foreach object, a Bloom filter representing all the column values, orproperties, that may be searched is created. The object and the Bloomfilter may be stored on datastore 124 as part of an index, configured,for example, as a database table. The index includes two further values:the Hamming value of the Bloom filter, and a value based on the binarylogarithm of the Bloom filter taken as an unsigned integer. These valuesidentify a subset of the stored Bloom filters, which may be viewedlogically as a region of datastore 124 containing the indexed Bloomfilters. The region may contain multiple unique Bloom filters. DuplicateBloom filters may also be present.

When searching the index for an exact match, the Hamming value and thebinary logarithm are used to locate a region of datastore 124 thatcontains the Bloom filter. All Bloom filters within the region aresearched and those that match exactly are selected. The matching objectsare further filtered to ensure that the desired properties are present.This filtering is necessary because Bloom filters may return falsepositives.

When looking for multiple objects, i.e., when a non-unique set ofproperties is selected, a target Bloom filter for the desired columnvalues is created. Regions corresponding to index values that aregreater than or equal to the values from the target Bloom filter arescanned for matching Bloom filters. As with the exact search, thematching objects are then filtered to ensure that the desired propertiesare actually present.

Database indexing program 112 operates generally to facilitate searchingfor database objects represented as Bloom filters, in accordance with anembodiment of the present invention. The database objects may reside,for example, on datastore 124. Database indexing program 112 may includesetup module 114, insert module 116, read module 118, scan module 120,and delete module 122.

Setup module 114 creates a table in a database on datastore 124, inaccordance with an exemplary embodiment of the present invention. Thetable stores a collection of Bloom filters with a common configuration,each representing a database object with specified properties, andserves to index the objects. The table includes columns for the Hammingvalue of the Bloom filter, the binary log of the Bloom filter, the Bloomfilter itself, and for a copy of the object or a representation of theobject such as a binary large object (BLOB) representing the object. ABLOB is a collection of binary data of variable length that is stored asa single entity in a database. BLOBs may be, for example, text, animage, an audio or multimedia object, etc. Setup module 114 may indexthe table, using standard methods of database indexing, by the columnscontaining the Hamming value and the binary log.

Alternatively, setup module 114 may augment an existing database tablewith columns for a Bloom filter representing each object in the table,the Hamming value of the Bloom filter, the binary log of the Bloomfilter, and the object or a representation of the object, such as aBLOB. Setup module 114 may index the augmented table, using standardmethods of database indexing, by the columns containing the Hammingvalue and the binary log.

In an embodiment of the invention, setup module 114 may select a Bloomfilter configuration, based on a predetermined number n of objectproperties and acceptable false positive probability p. The n objectproperties may include any object properties that are likely to besearched for. Based on n and p, setup module may use equation (1) todetermine a size m of a Bloom filter and a number k of hash functions.Setup module 114 may select a set of k independent hash functions, forexample from a predetermined family of hash functions. In otherembodiments, setup module 114 may use a predetermined Bloom filterconfiguration, including values for n, m, and k, and a predetermined setof k independent hash functions.

Insert module 116 operates to insert an object in the table created bysetup module 114, in accordance with an embodiment of the presentinvention. Insert module 116 creates a Bloom filter of size m for theobject. Insert module 116 hashes each of the n object properties ktimes, setting the appropriate bits in the Bloom filter to 1. Insertmodule 116 determines the Hamming value and the binary logarithm of theBloom filter. Insert module 116 creates a candidate row for the tablethat includes the Hamming value, the binary logarithm, the Bloom filter,and a representation of the object, such as a BLOB. Insert module 116searches the table for an exact match to the row, using an availabledatabase find/search instruction. If a match is found, the object isalready in the table and no further action is taken. If no match isfound, insert module 116 inserts the object in the table using anavailable database insert instruction.

Read module 118 operates to identify database objects that exactly matchspecified properties, in accordance with an embodiment of the presentinvention. In response to a request to identify database objects thatexactly match specified properties, read module 118 creates a targetBloom filter of size m for the specified properties and hashes each ofthe n properties k times, setting the appropriate bits in the Bloomfilter to 1. Read module 118 determines the Hamming value and the binarylogarithm of the target Bloom filter. Read module 118 searches the tablefor candidate Bloom filters, which have a Hamming value and a binarylogarithm that exactly match the Hamming value and binary logarithm ofthe target Bloom filter. For this purpose an available databasesearch/find instruction may be used. If no candidates are found, readmodule 118 may report ‘none found’. If one or more candidates are found,read module 118 filters them by verifying that the target Bloom filtermatches the candidate Bloom filters. Read module 118 returns the BLOB s(or other object representations) for which the Bloom filters match thetarget Bloom filter to the calling application, to verify that theproperties do, in fact, match the properties of the target Bloom filter,and to eliminate any matches that are false positives. Alternatively,read module 118 may eliminate the false positives.

Scan module 120 operates to identify database objects that havespecified properties when multiple property matches are expected, inaccordance with an embodiment of the present invention. Scan module 120creates a target Bloom filter of size m for the specified properties,and hashes each of the n properties k times, setting the appropriatebits in the Bloom filter to 1. Scan module 120 determines the Hammingvalue and the binary logarithm of the target Bloom filter. Scan module120 searches the table for candidate Bloom filters, which have Hammingvalues greater than or equal to the Hamming value of the target Bloomfilter and binary logarithms that are greater than or equal to thebinary logarithm of the target Bloom filter. For this purpose anavailable database search/find instruction may be used. If no candidatesare found, scan module 120 may report ‘none found’. If one or morecandidates are found, scan module 120 filters them by verifying that thetarget Bloom filter matches the candidate Bloom filters. Scan module 120returns the BLOB s for which the Bloom filters match to the callingapplication, to verify that the properties do in fact match theproperties of the target Bloom filter. This step is necessary becauseBloom filter matches may be false positives. Alternatively, scan module120 may eliminate the false positives. The difference between a scanoperation and a read operation is the inequality checks for the Hammingvalue and the binary logarithm.

For example, Table 1 illustrates an example database table of carproperties. The properties are Make, Type, and Color, and each car,identified by a unique ID, is associated with a Bloom filter of size 72,which is displayed in this example in hexadecimal form. The Bloomfilters might be, for example, the result of applying 17 hash functionsto the three properties, as described above. Two additional columns, onefor each of Hamming value and binary log of the Bloom filter, have beenadded to the database table to facilitate searches, in accordance withthe invention. Suppose that the table is to be scanned for the twoproperties make and type. To scan the table, a search key isconstructed, which includes Hamming value, binary log, and Bloom filter,as well as make and type. For example, a search key might be:

-   -   28, 69.63427734375, 344C032260487974AD, ‘Audi’, ‘Van’

TABLE 1 CAR TABLE ID HAMMING VALUE BINARY LOG BLOOM FILTER (HEX) MAKETYPE COLOR 0 41 69.82421875 3A7961E3C17D7C16F7 ‘VW’ ‘Van’ ‘red’ 1 3467.859375 0EE3A2C19FFD446424 ‘VW’ ‘Bug’ ‘blue’ 2 40 69.9531253EEBA2E3CF7C7C14A4 ‘VW’ ‘Van’ ‘blue’ 3 37 69.649414062534CE83A26E587D74AD ‘Audi’ ‘Van’ ‘blue’ 4 36 69.6357421875345C432260597976FF ‘Audi’ ‘Van’ ‘red’ 5 36 68.293945312514B72B11215D35EF0F ‘Audi’ ‘Sedan’ ‘white’ 6 31 66.20898437504D6A3902E5835670D ‘Audi’ ‘Sedan’ ‘blue’Audi is a registered trademark of AUDI AG. VW is a registered trademarkof Volkswagen AG. The result of the scan operation includes all rows inTable 1 with a Hamming value greater than or equal to the key Hammingvalue, 28, and a binary log greater than or equal to the search keybinary log, 69.63427734375. In this example, these are the rows with IDs0, 2, 3, and 4, as shown in Table 2:

TABLE 2 CAR TABLE SCAN RESULTS ID HAMMING VALUE BINARY LOG BLOOM FILTER(HEX) MAKE TYPE COLOR 0 41 69.82421875 3A7961E3C17D7C16F7 ‘VW’ ‘Van’‘red’ 2 40 69.953125 3EEBA2E3CF7C7C14A4 ‘VW’ ‘Van’ ‘blue’ 3 3769.6494140625 34CE83A26E587D74AD ‘Audi’ ‘Van’ ‘blue’ 4 36 69.6357421875345C432260597976FF ‘Audi’ ‘Van’ ‘red’After the results are filtered by Bloom filter matching via an ANDoperation, only the rows with IDs 3 and 4 remain.

Delete module 122 operates to delete database objects that havespecified properties, in accordance with an embodiment of the presentinvention. In response to a request to delete database objects thatexactly match specified properties, delete module 122 creates a targetBloom filter of size m for the specified properties, hashes each of then properties k times, and sets the appropriate bits in the target Bloomfilter to 1. Delete module 122 determines the Hamming value and thebinary logarithm of the target Bloom filter. Delete module 122 creates acandidate row for the table, including the Hamming value, the binarylogarithm, the Bloom filter, and the object representation, for example,a BLOB. Delete module 122 searches the table for exact matches to thecandidate row. For this purpose an available database search/findinstruction may be used. If no candidate is found, the object was notpresent in the table and delete module 122 may report ‘none found’. Ifone or more matches are found, delete module 122 removes the objectsfrom the table.

For databases that do not support numeric indexes, the Hamming value maybe a zero-prefixed string with the length of the string being the lengthm expressed as a string, and the binary logarithm may be expressed as aconcatenation of the position of the highest 1 bit in the Bloom filter,expressed with leading zeros as with the Hamming value, and a fixednumber of the next highest bits similarly expressed. In general, fivebits is sufficient.

Implementation has been described for relational databases, though usinguser-defined functions to perform the bitmap scanning on the databaserather than on the client is advisable. As will be apparent to skilledpractitioners, implementation for databases other than relationaldatabases is also straightforward, for example, using custom filters.Moreover, the same basic approach will work as well, with properstructuring, on a naked file system.

FIG. 2 is a flowchart depicting operational steps of setup module 114 indatabase indexing program 112, in accordance with an exemplaryembodiment of the invention. Setup module 114 selects or receives aBloom filter configuration (step 210). Setup module 114 creates adatabase table with columns for a Bloom filter, the Hamming value of theBloom filter, a binary log of the Bloom filter, and a representation ofa database object, for example a BLOB (step 220). Setup module 114indexes the database table according to the columns with Hamming valueand binary log (step 230).

FIG. 3 is a flowchart depicting operational steps of insert module 116in database indexing program 112, in accordance with an exemplaryembodiment of the invention. Insert module 116 receives a databaseobject (step 310). Insert module 116 creates a Bloom filter for theobject (step 320). Insert module 116 computes the Hamming value and abinary log of the Bloom filter (step 330). Insert module 116 inserts arow with the Bloom filter, Hamming value, binary log, and arepresentation of the object in the database table created by setupmodule 114 (step 340). Insert module 116 updates the index created bysetup module 114 (step 350).

FIG. 4 is a flowchart depicting operational steps of read module 118 indatabase indexing program 112, in accordance with an exemplaryembodiment of the invention. Read module 118 receives a set of objectproperties (step 410). Read module 118 creates a Bloom filter for theobject properties (step 420). Read module 118 computes the Hamming valueand a binary log of the Bloom filter (step 430). Read module 118searches the database table created by setup module 114 for rows thathave a Hamming value equal to the computed Hamming value and a binarylog equal to the computed binary log (step 440). Read module 118 filtersthe rows found to select those with a Bloom filter that matches thetarget Bloom filter (step 450). Read module 118 returns any objects thathave matching Bloom filters (step 460).

FIG. 5 is a flowchart depicting operational steps of scan module 120 indatabase indexing program 112, in accordance with an exemplaryembodiment of the invention. Scan module 120 receives a set of objectproperties (step 510). Scan module 120 creates a Bloom filter for theobject properties (step 520). Scan module 120 computes the Hamming valueand a binary log of the Bloom filter (step 530). Scan module 120searches the database table created by setup module 114 for rows thathave a Hamming value greater than or equal to the computed Hamming valueand a binary log greater than or equal to the computed binary log (step540). Scan module 120 filters the rows found to select those with aBloom filter that matches the target Bloom filter (step 550). Scanmodule 120 returns any objects that have matching Bloom filters (step560).

FIG. 6 is a flowchart depicting operational steps of delete module 122in database indexing program 112, in accordance with an exemplaryembodiment of the invention. Delete module 122 receives a set of objectproperties (step 610). Delete module 122 creates a Bloom filter for theobject properties (step 620). Delete module 122 computes the Hammingvalue and a binary log of the Bloom filter (step 630). Delete module 122creates a candidate row with the target Bloom filter, the Hamming value,the binary log, and the object (step 640). Delete module 122 searchesthe database table created by setup module 114 for rows that exactlymatch the candidate row (step 650). Delete module 122 deletes any rowsthat match (step 660).

FIG. 7 depicts a block diagram 900 of components of computing device 110(FIG. 1), in accordance with an embodiment of the present invention. Itshould be appreciated that FIG. 7 provides only an illustration of oneimplementation and does not imply any limitations with regard to theenvironments in which different embodiments may be implemented. Manymodifications to the depicted environment may be made.

Computing device 110 may include one or more processors 902, one or morecomputer-readable RAMs 904, one or more computer-readable ROMs 906, oneor more computer readable storage media 908, device drivers 912,read/write drive or interface 914, network adapter or interface 916, allinterconnected over a communications fabric 918. Communications fabric918 may be implemented with any architecture designed for passing dataand/or control information between processors (such as microprocessors,communications and network processors, etc.), system memory, peripheraldevices, and any other hardware components within a system.

One or more operating systems 910, and one or more application programs928, for example, database indexing program 112 (FIG. 1), are stored onone or more of the computer readable storage media 908 for execution byone or more of the processors 902 via one or more of the respective RAMs904 (which typically include cache memory). In the illustratedembodiment, each of the computer readable storage media 908 may be amagnetic disk storage device of an internal hard drive, CD-ROM, DVD,memory stick, magnetic tape, magnetic disk, optical disk, asemiconductor storage device such as RAM, ROM, EPROM, flash memory orany other computer-readable tangible storage device that can store acomputer program and digital information.

Computing device 110 may also include a R/W drive or interface 914 toread from and write to one or more portable computer readable storagemedia 926. Application programs 928 on client computing devices and/orcomputing device 110 may be stored on one or more of the portablecomputer readable storage media 926, read via the respective R/W driveor interface 914 and loaded into the respective computer readablestorage media 908.

Computing device 110 may also include a network adapter or interface916, such as a TCP/IP adapter card or wireless communication adapter(such as a 4G wireless communication adapter using OFDMA technology).Application programs 928 on computing device 110 may be downloaded tothe computing device from an external computer or external storagedevice via a network (for example, the Internet, a local area network orother wide area network or wireless network) and network adapter orinterface 916. From the network adapter or interface 916, the programsmay be loaded onto computer readable storage media 908. The network maycomprise copper wires, optical fibers, wireless transmission, routers,firewalls, switches, gateway computers and/or edge servers.

Computing device 110 may also include a display screen 920, a keyboardor keypad 922, and a computer mouse or touchpad 924. Device drivers 912interface to display screen 920 for imaging, to keyboard or keypad 922,to computer mouse or touchpad 924, and/or to display screen 920 forpressure sensing of alphanumeric character entry and user selections.The device drivers 912, R/W drive or interface 914 and network adapteror interface 916 may comprise hardware and software (stored on computerreadable storage media 908 and/or ROM 906).

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include anon-transitory computer readable storage medium (or media) havingcomputer readable program instructions thereon for causing a processorto carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The programs described herein are identified based upon the applicationfor which they are implemented in a specific embodiment of theinvention. However, it should be appreciated that any particular programnomenclature herein is used merely for convenience, and thus theinvention should not be limited to use solely in any specificapplication identified and/or implied by such nomenclature.

The foregoing description of various embodiments of the presentinvention has been presented for purposes of illustration anddescription. It is not intended to be exhaustive nor to limit theinvention to the precise form disclosed. Many modifications andvariations are possible. Such modification and variations that may beapparent to a person skilled in the art of the invention are intended tobe included within the scope of the invention as defined by theaccompanying claims.

What is claimed is:
 1. A computer-implemented method for indexingobjects in a datastore, wherein the objects are characterized by a setof object properties, the method comprising: for one or more objectsfrom a datastore: creating, by a computer, a Bloom filter for the objectproperties, based on a predetermined Bloom filter configuration;determining, by the computer, a Hamming value of the Bloom filter;calculating, by the computer, a binary logarithm of the Bloom filter;and storing, by the computer, the Bloom filter, the Hamming value, thebinary logarithm, and a representation of the object in the datastore.2. A method in accordance with claim 1, further comprising: providing,by the computer, a set of objects from the datastore in response to arequest to search the datastore for objects with specified properties.3. A method in accordance with claim 2, wherein the providing comprises:creating, by a computer, a search Bloom filter for the objectproperties, based on the predetermined Bloom filter configuration;determining, by the computer, a Hamming value of the search Bloomfilter; calculating, by the computer, a binary logarithm of the searchBloom filter; identifying, by the computer, objects in the datastorewhich have respective Hamming values equal to the Hamming value of thesearch Bloom filter and respective binary logarithms equal to the binarylogarithm of the search Bloom filter; identifying, by the computer, fromthe identified objects, ones whose Bloom filters match the search Bloomfilter, wherein a Bloom filter matches the search Bloom filter if each 1bit in the search Bloom filter corresponds to a 1 bit at the sameposition in the Bloom filter; and eliminating, by the computer, from theidentified objects whose Bloom filters match the search Bloom filter,false positives.
 4. A method in accordance with claim 1, furthercomprising: in response to receiving a request to delete objects in thedatastore with specified properties: creating, by a computer, a searchBloom filter for the object properties, based on the predetermined Bloomfilter configuration; determining, by the computer, a Hamming value ofthe search Bloom filter; calculating, by the computer, a binarylogarithm of the search Bloom filter; identifying, by the computer,objects in the datastore which have respective Hamming values equal tothe Hamming value of the search Bloom filter and respective binarylogarithms equal to the binary logarithm of the search Bloom filter;identifying, by the computer, from the identified objects, ones whoseBloom filters match the search Bloom filter, wherein a Bloom filtermatches the search Bloom filter if each 1 bit in the search Bloom filtercorresponds to a 1 bit at the same position in the Bloom filter;eliminating, by the computer, from the identified objects whose Bloomfilters match the search Bloom filter, false positives; and deleting, bythe computer, objects from the identified objects whose Bloom filtersmatch the search Bloom filter any that are not false positives.
 5. Amethod in accordance with claim 1, wherein the predetermined Bloomfilter configuration is selected based on a predetermined number ofobject properties and on a predetermined false positive probability. 6.A method in accordance with claim 1, wherein the representation of theobject is one of: a copy of the object, a binary large object, or areference to the object.
 7. A method in accordance with claim 1, whereinthe datastore is a relational database, and wherein the objectscorrespond to rows in a database table, and wherein the Bloom filter,the Hamming value, the binary logarithm, and the representation of theobject in the datastore correspond to columns in the database table, themethod further comprising: indexing, by the computer, the objects byHamming value and binary logarithm.
 8. A computer system for indexingobjects in a datastore, wherein the objects are characterized by a setof object properties, the computer system comprising: one or morecomputer processors, one or more non-transitory computer-readablestorage media, and program instructions stored on one or more of thecomputer-readable storage media for execution by at least one of the oneor more processors, the program instructions comprising: programinstructions, for one or more objects from a datastore, to: create aBloom filter for the object properties, based on a predetermined Bloomfilter configuration; determine a Hamming value of the Bloom filter;calculate a binary logarithm of the Bloom filter; and store the Bloomfilter, the Hamming value, the binary logarithm, and a representation ofthe object in the datastore.
 9. A computer system in accordance withclaim 8, further comprising: program instructions to provide a set ofobjects from the datastore in response to a request to search thedatastore for objects with specified properties.
 10. A computer systemin accordance with claim 9, wherein program instructions to providecomprise: program instructions to create a search Bloom filter for theobject properties, based on the predetermined Bloom filterconfiguration; program instructions to determine a Hamming value of thesearch Bloom filter; program instructions to calculate a binarylogarithm of the search Bloom filter; program instructions to identifyobjects in the datastore which have respective Hamming values equal tothe Hamming value of the search Bloom filter and respective binarylogarithms equal to the binary logarithm of the search Bloom filter;program instructions to identify from the identified objects, ones whoseBloom filters match the search Bloom filter, wherein a Bloom filtermatches the search Bloom filter if each 1 bit in the search Bloom filtercorresponds to a 1 bit at the same position in the Bloom filter; andprogram instructions to eliminate from the identified objects whoseBloom filters match the search Bloom filter, false positives.
 11. Acomputer system in accordance with claim 8, further comprising: inresponse to receiving a request to delete objects in the datastore withspecified properties: program instructions to create a search Bloomfilter for the object properties, based on the predetermined Bloomfilter configuration; program instructions to determine a Hamming valueof the search Bloom filter; program instructions to calculate a binarylogarithm of the search Bloom filter; program instructions to identifyobjects in the datastore which have respective Hamming values equal tothe Hamming value of the search Bloom filter and respective binarylogarithms equal to the binary logarithm of the search Bloom filter;program instructions to identify from the identified objects, ones whoseBloom filters match the search Bloom filter, wherein a Bloom filtermatches the search Bloom filter if each 1 bit in the search Bloom filtercorresponds to a 1 bit at the same position in the Bloom filter; programinstructions to eliminate, from the identified objects whose Bloomfilters match the search Bloom filter, false positives; and programinstructions to delete objects, from the identified objects whose Bloomfilters match the search Bloom filter, any that are not false positives.12. A computer system in accordance with claim 8, wherein thepredetermined Bloom filter configuration is selected based on apredetermined number of object properties and on a predetermined falsepositive probability.
 13. A computer system in accordance with claim 8,wherein the representation of the object is one of: a copy of theobject, a binary large object, or a reference to the object.
 14. Acomputer system in accordance with claim 8, wherein the datastore is arelational database, and wherein the objects correspond to rows in adatabase table, and wherein the Bloom filter, the Hamming value, thebinary logarithm, and the representation of the object in the datastorecorrespond to columns in the database table, the computer system furthercomprising: instructions to index the objects by Hamming value andbinary logarithm.
 15. A computer program product for indexing objects ina datastore, wherein the objects are characterized by a set of objectproperties, the computer program product comprising: one or morenon-transitory computer-readable storage media and program instructionsstored on the one or more computer-readable storage media, the programinstructions comprising: program instructions, for one or more objectsfrom a datastore, to: create a Bloom filter for the object properties,based on a predetermined Bloom filter configuration; determine a Hammingvalue of the Bloom filter; calculate a binary logarithm of the Bloomfilter; and store the Bloom filter, the Hamming value, the binarylogarithm, and a representation of the object in the datastore.
 16. Acomputer program product in accordance with claim 15, furthercomprising: program instructions to provide a set of objects from thedatastore in response to a request to search the datastore for objectswith specified properties.
 17. A computer program product in accordancewith claim 16, wherein program instructions to provide comprise: programinstructions to create a search Bloom filter for the object properties,based on the predetermined Bloom filter configuration; programinstructions to determine a Hamming value of the search Bloom filter;program instructions to calculate a binary logarithm of the search Bloomfilter; program instructions to identify objects in the datastore whichhave respective Hamming values equal to the Hamming value of the searchBloom filter and respective binary logarithms equal to the binarylogarithm of the search Bloom filter; program instructions to identifyfrom the identified objects, ones whose Bloom filters match the searchBloom filter, wherein a Bloom filter matches the search Bloom filter ifeach 1 bit in the search Bloom filter corresponds to a 1 bit at the sameposition in the Bloom filter; and program instructions to eliminate fromthe identified objects whose Bloom filters match the search Bloomfilter, false positives.
 18. A computer program product in accordancewith claim 15, further comprising: in response to receiving a request todelete objects in the datastore with specified properties: programinstructions to create a search Bloom filter for the object properties,based on the predetermined Bloom filter configuration; programinstructions to determine a Hamming value of the search Bloom filter;program instructions to calculate a binary logarithm of the search Bloomfilter; program instructions to identify objects in the datastore whichhave respective Hamming values equal to the Hamming value of the searchBloom filter and respective binary logarithms equal to the binarylogarithm of the search Bloom filter; program instructions to identifyfrom the identified objects, ones whose Bloom filters match the searchBloom filter, wherein a Bloom filter matches the search Bloom filter ifeach 1 bit in the search Bloom filter corresponds to a 1 bit at the sameposition in the Bloom filter; program instructions to eliminate, fromthe identified objects whose Bloom filters match the search Bloomfilter, false positives; and program instructions to delete objects,from the identified objects whose Bloom filters match the search Bloomfilter, any that are not false positives.
 19. A computer program productin accordance with claim 15, wherein the predetermined Bloom filterconfiguration is selected based on a predetermined number of objectproperties and on a predetermined false positive probability.
 20. Acomputer program product in accordance with claim 15, wherein thedatastore is a relational database, and wherein the objects correspondto rows in a database table, and wherein the Bloom filter, the Hammingvalue, the binary logarithm, and the representation of the object in thedatastore correspond to columns in the database table, the computerprogram product further comprising: instructions to index the objects byHamming value and binary logarithm.